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ABSTRACT 

This  paper  deals  with  the  problem  of  estimacing  simultaneously  the 
parameters  (cell  probabilities)  of  m  >  2  independent  multinomial  distri¬ 
butions,  with  respect  to  a  quadratic  loss  function.  An  empirical  Bayes 
estimator  is  given  which  is  shown  to  have  smaller  risk  than  the  maximum 
likelihood  estimator  for  sufficiently  large  values  of  m,  at  all  points  of 
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1.  INTRODUCTION 


The  multinomial  distribution  arises  in  various  statistical  analyses. 
In  this  paper  we  consider  the  problem  of  estimating  simultaneously  the 
parameters  (cell  probabilities)  of  m  >_  2  independent  multinomial  distri¬ 
butions.  A  new  estimator  is  obtained  from  empirical  Bayes  considerations, 
where  the  estimate  of  the  parameters  of  one  distribution  depends  on  the 
observations  taken  from  the  other  distributions.  We  compare  the  new  esti¬ 
mator  with  the  maximum  likelihood  estimator  (MLE)  and  show  that  for  large 
values  of  m  the  risk  of  the  new  estimator  is  smaller  than  the  risk  of  the 
MLE  at  all  points  excluding  a  small  subset  of  the  parameter  space. 

The  problem  of  estimating  the  parameters  of  a  multinomial  distribu¬ 
tion  has  been  considered  by  various  authors.  Alam  (1979)  and  Johnson 
(1971)  have  shown  that  for  ra  =  1  the  MLE  is  admissible  with  respect  to  the 
sum  of  squared  errors  as  the  loss  function.  The  estimation  of  multinomial 
probabilities  has  been  considered  from  a  decision  theoretic  point  of  view 
by  Steinhaus  (1957),  Trybula  (1958)  and  Rutkowska  (1977).  In  a  recent 
paper  Olkin  and  Sobel  (1979)  have  discussed  the  problem  of  estimating  the 
parameters  of  m  ^  2  binomial  distributions. 

In  Section  2  we  introduce  the  new  empirical  Bayes  estimator.  In  Sec¬ 
tion  3  we  compare  the  risk  of  the  empirical  Bayes  estimator  with  the  MLE. 

2.  EMPIRICAL  BAYES  ESTIMATOR 


Let  ■  (x^, . . .  ,x^) ,  i  ■  l,...,m  denote  independent  observations 

K 

from  ra  multinomial  distributions,  each  with  K  >  2  cells,  where  }.  ,x,,  * 

-  *-j-l  ij 

Let  p^  *  (p^i» . . . *PiK)  denote  the  vector  of  cell  probabilities  associated 

with  the  ith  distribution,  where  p . .  >  0,  1  *  1,...,K  and  7,  ,p.±  *  1. 

ij  J  ‘•j*!  ij 
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For  estimating  the  cell  probabilities  simultaneously,  let  the  loss  function 
be  given  by 

«*•»  ■  ■  I  JL'w2 

where  5^  denotes  the  estimate  of  p^  ,  <5  ■  (d^)  and  P  *  (p^). 

First  we  obtain  a  Bayes  estimator  of  P,  as  follows.  Suppose  that  the 
parameters  of  the  m  distributions  are  distributed  a  priori,  independently 
and  identically  according  to  the  Dirichlet  distribution,  given  by  the  den¬ 
sity  function 


<ru» 


where  X  is  a  positive  number,  0  <  0^  <  1,  j  ■  1,...,K  and  £^^0^  *  1. 

A  Bayes  estimator  of  P  with  respect  to  the  Dirichlet  prior  distribution  is 

A  A 

given  by  <5  »  (6^),  where 


5ij  *  (*1:j+X)/(n+KX) . 


(2.1) 


Let  q.  »  1  -  T.  .pf.,  q  *  ~  I?  ,q.  and  y  *  I?  .xt..  The  risk  of 
Mi  ‘,j»lrij’  ra  ^i*!1!  LimlLjml  ij 


S  is  given  by 


R(<$*)  »  [(n-X2K2)q  +  K(K-l)A2Hn+KA)"2. 


(2.2) 


The  value  of  X  minimizing  R(5  )  is  equal  to  X^,  given  by 


XQ  -  q(K-l-Kq) 


(2.3) 


*  * 

Let  R  denote  the  value  of  R(5  )  for  X  ■  Aq.  We  have 


(2.4) 
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where  5^*  denotes  the  MLE,  given  by 
«  2 

q  is  given  by  q  *  (mn  -y)/mn(n-l). 
estimated  value  of  A,  given  by 


■  x^/n.  An  unbiased  estimator  of 
Substituting  q  for  q  in  (2.3)  we  get  an 


A  -  q(K-l-Kq)-1  (2.5) 

-  <»  for  q  *  (K-l)/K. 


Substituting  A  for  A  in  (2.1),  we  obtain  an  empirical  Bayes  estimator  6 
of  P,  given  by 

^ij  =  (xi:j+*)/(n+K*>  (2.6) 

■  1/K  for  A  *  ®. 


The  quantity  q^  is  a  measure  of  diversity  of  the  ith  multinomial  distri¬ 
bution.  It  is  equal  to  zero  when  the  probability  mass  of  the  distribution  is 
concentrated  in  a  single  cell.  It  achieves  its  maximum  value  equal  to  (K-l)/K. 
when  the  probability  mass  is  equally  distributed  among  the  K  cells  of  the 
distribution.  The  value  of  q  represents  the  average  diversity  of  the  m  mul¬ 
tinomial  distributions.  In  the  following  section  we  show  that  the  empirical 

Bayes  estimator  <5  has  smaller  risk  than  the  MLE  for  sufficiently  large  values 
2 

of  mq  .  Therefore,  in  practice  when  the  parameters  of  a  large  number  of  mul¬ 
tinomial  distributions  are  simultaneously  estimated  for  which  the  average 
diversity  is  unlikely  to  be  very  small,  the  empirical  Bayes  estimator  should 
be  preferred  to  the  MLE. 

The  preceding  observation  is  supported  by  the  numerical  results  shown 

in  Table  1  below.  The  table  gives  the  empirical  values  of  R(6),  R(6®)  and 
2 

mq  for  certain  values  of  m,  n,  K  and  randomly  chosen  values  of  P.  The  value 
-  0 

of  R(<$)  and  R ( 5  )  for  any  fixed  value  of  P  was  obtained  from  simulation  by 

-  0 

the  Monte  Carlo  method.  It  is  seen  from  the  table  that  R(<$)  <  R(5  )  in  all 


Che  observed  cases  except  one  when  m  *  5,  n  *  10,  K  *  2  for  which  R(«)  - 

.0247  and  R(5°)  -  .0241.  This  case  corresponds  to  the  smallest  value  of 

2 

mq  *  0.2856  given  in  the  table. 


3.  RISK  OF  6 


From  (2.2)  it  is  seen  that  R(<S  )  first  decreases  then  increases  as  X 

varies  from  0  to  ®.  It  is  eoual  to  and  — - —  —  q  for  X  =  0  and  °>,  respec- 

n  K. 

tively.  Hence,  R(<5*)  <  R(S°)  for  all  positive  values  of  X  if  ^  ^ -  q. 

Therefore,  we  shall  assume  that 


„  <  n(K-l) 
q  -  K(n+1)  ' 


(3.1) 


The  value  of  R(6)  for  large  m  is  given  as  follows:  From  (2.3)  and 


(2.5)  we  get 


X  -  X 


(K-l) (q-q) 


0  (K-l-Kq) (K-l-Kq)  * 


(3.2) 


Now,  Eq  -  q  and  Var(q)  <  by  Lemma  3.1  below.  Therefore,  for  large  m 


EX  -  XQ  +  [K(K-i)/ (K-l-Kq) 2]Var(q)  +  o(^) 
F.(X-XQ)2  =■  [(K-l)2/ (K-l-Kq) 4]Var(q)  +  o(i). 


and  (3.3) 
(3.4) 


The  value  of  R(6)  as  obtained  from  (2.6)  by  direct  computation,  is 
given  after  simplification  by 


R(5)  -  R*  +  nK(n+KX0)'2E(n+KX)"2(n(K-l)-K(n-l)q)(X-X0)2  (3.5) 


(K-l)X  +(n-l)q-(n+KA  )q 
+  2nK(n+KXQ)"  E(n+KX)~  (X-XQ)  [  - -  +  D 


where  Z  -  ±  %ml  1*^  (x^-np^) .  Now  EZ  -  0  and 


K _ c_ 

2n2(K-l)  mq 

Therefore,  we  have  the  following  result. 

Theorem  3.1.  The  empirical  Bayes  estimator  6  has  smaller  risk  than 

0  ° 
the  maximum  likelihood  estimator  6  for  sufficiently  large  values  of  mq“. 

Remark.  It  has  been  mentioned  in  the  introduction  that  the  MLE  is 

admissible  with  respect  to  the  quadratic  loss  function  when  m  *  1.  rt  is 

also  noted  by  Johnson  (1971)  that  the  admissibility  of  the  MLE  is  essentially 

for  the  reason  that  its  risk  is  small  near  the  boundary  of  the  parameter 


6 


space,  given  by  q  =  0.  Therefore,  the  condition  of  Theorem  3.1  that  the 
value  of  q  should  be  sufficiently  large  for  the  risk  of  5  to  be  smaller 
than  the  risk  of  3^  is  consistent  with  that  observation.  Johnson  has  also 
noted  that  there  is  no  Stein-effect  for  m  >  1,  that  is,  the  MLE  cannot  be 
improved  upon  by  letting  the  estimate  of  parameters  of  one  distribution  de¬ 
pend  on  the  observations  taken  from  the  other  distributions.  Therefore, 

-  0 
we  do  not  expect  that  5  has  a  uniformly  smaller  risk  than  6  . 

Finally,  we  give  the  lemma  which  was  used  in  the  proof  of  Theorem  3.1. 

Lemma  3.1.  Var(q)  <  v 

-  ^  —  m(n-l) 

Proof :  We  have 


K  ry  ^  .  K  i. 

Var(  l  x2  )  =  E(  J  x2  )2  -  (E  J  x2  )‘ 
ja=l  j=l  J  J 

<  (E  l  x2  )(n2-E  l  x2) 
j=l  j*l 


2,  2  _  r  2  , 

±  n  (n  -E  l  x  ) 

3-1  J 


n  (n-Dq^,. 


Summing  the  above  inequality  for  i  =  l,...,m  we  have 

3 

Var(y)  ^  mn  (n-l)q. 


Therefore 


Var(q)  <_ 


n(l  ,  . 

m(n-l) 


(3.9) 
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